Marc Deisenroth

Thursday 4th July 2019

Time:1.30pm

Ground Floor Seminar Room

25 Howland Street, London, W1T 4JG

Data-Efficient Reinforcement Learning

On our path toward fully autonomous systems, i.e., systems that operate in the real world without significant human intervention, reinforcement learning (RL) is a promising framework for learning to solve problems by trial and error. While RL has had many successes recently, a practical challenge we face is its data inefficiency: In real-world problems (e.g., robotics) it is not always possible to conduct millions of experiments, e.g., due to time or hardware constraints. In this talk, I will outline three approaches that explicitly address the data-efficiency challenge in reinforcement learning using probabilistic models. First, I will give a brief overview of a model-based policy search RL algorithm that can learn from small datasets. Second, I will describe an approach based on model predictive control that allows us to learn even faster while taking care of state or control constraints, which is important for safe exploration. Finally, I will introduce an approach to meta learning (in the context of model-based RL) based on latent variables, which allows us to speed up learning by generalizing learned concepts to new tasks.

Bio:

Marc Deisenroth is a Senior Lecturer at the Department of Computing, Imperial College London. Since September 2016, Marc has also been an advisor to PROWLER.io, a Cambridge-based startup. Marc's research interests center around data-efficient and autonomous machine learning. Marc was Program Chair of EWRL 2012, Workshops Chair of RSS 2013 and received Best Paper Awards at ICRA 2014 and ICCAS 2016. In 2018, Marc has been awarded The President's Award for Outstanding Early Career Researcher. He is a recipient of a Google Faculty Research Award and a Microsoft PhD Grant. Marc is a co-leader of the Machine Learning Initiative at Imperial, co-initiator of the AI@Imperial Network of Excellence and the Director of the Machine Learning Lab in Imperial's Data Science Institute. Marc co-organizes the Machine Learning Summer School 2019 in London with Arthur Gretton. In 2018, Marc spent half a sabbatical at the African Institute for Mathematical Sciences (Rwanda), where he taught a course on Foundations of Machine Learning as part of the African Masters in Machine Intelligence. He has co-authored the book "Mathematics for Machine Learning" (https://mml-book.com), to be published by Cambridge University Press in January 2020..